Journals
  Publication Years
  Keywords
Search within results Open Search
Please wait a minute...
For Selected: Toggle Thumbnails
Dynamic evaluation method for benefit of modality augmentation
Yizhen BI, Huan MA, Changqing ZHANG
Journal of Computer Applications    2023, 43 (10): 3099-3106.   DOI: 10.11772/j.issn.1001-9081.2022101510
Abstract142)   HTML13)    PDF (2673KB)(60)       Save

Focused on the difficulty and big benefit difference in acquiring new modalities, a method for dynamically evaluating benefit of modality augmentation was proposed. Firstly, the intermediate feature representation and the prediction results before and after modality fusion were obtained through the multimodal fusion network. Then, the confidence before and after fusion were obtained by introducing the True Class Probability (TCP) of two prediction results to confidence estimation. Finally, the difference between two confidences was calculated and used as an sample to obtain the benefit brought by the new modality. Extensive experiments were conducted on commonly used multimodal datasets and real medical datasets such as The Cancer Genome Atlas (TCGA). The experimental results on TCGA dataset show that compared with the random benefit evaluation method and the Maximum Class Probability (MCP) based method, the proposed method has the accuracy increased by 1.73 to 4.93 and 0.43 to 4.76 percentage points respectively, and the Effective Sample Rate (ESR) increased by 2.72 to 11.26 and 1.08 to 25.97 percentage points respectively. It can be seen that the proposed method can effectively evaluate benefits of acquiring new modalities for different samples, and has a certain degree of interpretability.

Table and Figures | Reference | Related Articles | Metrics